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Abstract — This paper explores the properties of adaptive sys- 
tems with closed-loop reference models. Using additional design 
freedom available in closed-loop reference models, we design new 
adaptive controllers that are (a) stable, and (b) have improved 
transient properties. Numerical studies that complement theoret- 
ical derivations are also reported. 



I. Introduction 

The central element of any adaptive systems is online 
parameter adjustment. This is usually accomplished by having 
a plant, determined by a dynamic model, along with a con- 
troller with adaptive parameters designed to compensate for 
the plant's actions, follow a reference model. The resulting 
error between the reference model and the plant is used to 
adjust the adaptive parameter. 

By definition, open-loop reference models are independent 
of the system dynamics. Such reference models have been the 
backbone of adaptive control for the past four decades 0, 
where modifications to the adaptive control law were first 
added for stability in the presence of bounded disturbances 
0-0 and semi-global stability in the presence of unmodeled 
dynamics 0, 0. We denote the underlying open-loop sys- 
tems in all these cases as Open-loop Reference Model (ORM)- 
adaptive systems. 

Earlier developments of adaptive systems included explo- 
rations of various kinds of reference models. The overall 
goal behind the selection of a reference model is that the 
corresponding tracking error must asymptotically decay in 
the absence of parametric uncertainties in the plant being 
controlled. In order to accomplish this goal, modifications 
of the open-loop reference models were explored 0, 0. 
Some of these modifications retained stability properties and 
were otherwise indistinguishable from ORM-adaptive systems 
and as a result, not pursued. Others could not be shown to 
be stable and were therefore dropped. Recently, a class of 
Closed-loop Reference Models (CRM) have been proposed 
for control of plants with unknown parameters whose states 
are accessible (see for example irT0l - lfT4l ) all of which are 
guaranteed to be stable and additionally portray improved 
transient performance. 
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Transient performance has been directly addressed in |fl6l 
and more recently in ITTll . |fT3l - |fT31 . The results in ifTTIl 
discussed the tracking error, but focused the attention mainly 
on the initial interval where the CRM-adaptive system exhibits 
fast time-scales. Results in |[T3) . lfT4l focus on deriving a 
damping ratio and natural frequency for adaptive systems 
with CRM. However, assumptions are made that the initial 
state error is zero and that the closed-loop system state is 
independent of the feedback gain in the reference model, both 
of which may not hold in general. The results in 05) too 
assume that initial state errors are zero. And in addition, the 
bounds derived in 03) are based upon norms, which 
do not capture the transient properties of adaptive systems. 
The results in ifTBI pertain to transient properties of adaptive 
systems, and quantify them using an £2 norm. The adaptive 
systems in question however are indirect, and do not pertain 
to CRMs. With the exception of 03), 03), none of the others 
have focused on derivatives of signals in the adaptive system, 
which is another measure of transient performance. 

Our focus in this is paper on CRM-based adaptive systems. 
Similar to ifTTI . 03), lTT4Tl we demonstrate their stability 
properties. Unlike these papers, we discuss transient properties 
of these adaptive systems using an £2 norm of error signals 
and derivatives of key signals such as adaptive parameters and 
the control input. These metrics are used to compare the CRM 
adaptive systems with their ORM counterparts 

Another class of adaptive systems that have been explored 
in the past where a noticeable improvement in transient perfor- 
mance was obtained is in the context of Combined/composite 
direct and indirect Model Reference Adaptive Control (CM- 
RAC) Q3, 0S1- While the results of these papers estab- 
lished stability of combined schemes, no rigorous guarantees 
of improved transient performance were provided, and have 
remained a conjecture OS). We focus on this class of adaptive 
systems as well in this paper and introduce CRMs into the 
picture. The resulting CMRAC with Closed-loop reference 
models (CMRAC-C) are shown to be stable, enable the feed- 
back of noise-free state estimates while guaranteeing stability, 
and most importantly are shown to have guaranteed transient 
properties. 

The main contributions of the paper are (i) direct adap- 
tive control structures with guaranteed transient performance, 
(ii) combined direct and indirect adaptive controllers with 
guaranteed transient performance, and (iii) the development 
of adaptive systems that allow feedback from noise free 
regressors. These are realized by using the extra degree of 
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freedom available in the CRM in terms of a feedback gain, and 
by exploiting exponential convergence properties of the CRM- 
adaptive system. The latter is made possible by introducing 
a projection algorithm with a known upper bound on the 
unknown parameters. 

Previous work by the same authors on this subject can be 
found in ||2T1 - Il23l . The findings presented in [21 1 are pre- 
liminary and illustrate the waterbed affect through Euclidean 
bounds of the rate of control input at specific times of interest. 
The results presented in l22l . l23l are a condensed version 
of this article and do not contain all of the discussions or 
proofs necessary to have a clear self contained presentation of 
this material. While much of what is presented in the paper 
is restricted to plants with state-variables accessible, the same 
idea can be extended to adaptive control using output feedback 
and is the topic of current investigation |2()|. 

The results in this paper are organized as follows: Section II 
introduces the basic structure of CRM adaptive control as well 
as the Projection Operator. Section III investigates the transient 
response of CRM. Section IV investigates the robustness 
properties of CRM adaptive control. Section V contains the 
stability analysis of CMRAC-C. Section VI analyses the 
transient performance of CMRAC-C. Section VII contains 
the analysis of CMRAC-CO and its robustness properties in 
regard to measurement noise. Section VIII compares CRM, 
CMRAC-C and CMRAC-CO structures. Section IX contains 
our concluding remarks. 

II. The CRM-Adaptive System 

In this section, we describe the CRM-adaptive system, and 
establish its stability and convergence properties in the absence 
of any perturbations other than parametric uncertainties. We 
first describe the CRM-adaptive system and prove its closed- 
loop stability. After some preliminaries on matrix bounds, we 
introduce a projection algorithm in the adaptive law. This is 
used to derive exponentially converging bounds on the key 
variables in the CRM-adaptive system. 

Consider the linear system dynamics with scalar input 

x(t) = A p x(t) + bu(t) (1) 

where x(t) £ R™ is the state vector, u(t) € R is the control 
input, A p £ R™ x ™ is unknown and b £ R™ is known. Our 
goal is to design the control input such that x(t) follows the 
reference model state x m (t) £ R ra defined by the following 
dynamics 

(t) = A (t) + br(t)-L(x(t)-x m (t)) (2) 

where A m £ R™ x " is Hurwtitz and r(t) £ R is a bounded pos- 
sible time varying reference command. L £ R" x " is denoted 
as the Luenberger-gain, and is chosen such that 

A~m = A m + L (3) 

is Hurwitz. Equation (O is referred to as a CRM, and when 
L = the classical ORM is recovered. 

Assumption 1. A parameter vector 9* £ R™ exists that satis- 
fies the matching condition 

A rn = A P + W* T . (4) 



The control input is chosen in the form 

u(t) = 9 T {t)x{t) +r(t) (5) 

where 9(t) £ R" is the adaptive control gain with the update 
law 

9(t) = -Tx{t)e T {t)Pb (6) 

with T = T T > 0, e(i) = x(t) — x m (t) is the model following 
error and P = P T > is the solution to the algebraic 
Lyapunov equation 

A-^P + PA~m — "4xn- (7) 

The underlying error model in this case is given by 

e(t)=A m e{t) + W(t)x(t) (8) 

where 9(t) = 9(t) — 9* is the parameter error. 

Theorem 1. The closed-loop adaptive system with (HJ, (fJJ, 
(0 and (O is globally stable with e(t) tending to zero 
asymptotically, under the matching condition in ©. 

Proof: It is straight forward to show using (O and (© 

that 

V(e,9) = e T Pe + 9 T T- 1 9 (9) 

is a Lyapunov function. Since e is bounded, the structure of 
(01 implies that x m is bounded, x in turn and u are bounded. 
Barbalat lemma ensures asymptotic convergence of e(t) to 
zero. □ 

Corollary 2. For all e > there exists T(e, L) > such that 
t > T(e,L) implies ||e(£)|| < e. 

The above corollary seems redundant. It is however used later 
in the transient performance. 

The overall CRM-adaptive system is defined by ([TJ, ©, 
(O, and ©. The standard open-loop reference model is given 
by 

x° m {t) = A m x° m (t) + br(t) (10) 
with the corresponding tracking error 

e°(t)=x(t)-x° m (t). (11) 

One can in fact view the error e° as the true tracking error 
and e as a pseudo-tracking error. The question that arises 
is whether the convergence properties that are assured in an 
ORM-adaptive system, of e°(t) tending to zero is guaranteed 
in a CRM-adaptive system as well. This is addressed in the 
following corollary: 

Corollary 3. The state vector x(t) converges to as 

t — > oo. 

Proof: From Theorem [T] we can conclude that e(t) — > 
asymptotically. Thus we can conclude that x m (t) — » x^t) 
as e(t) — s- 0, implying that e° —> 0, thus x(t) — > £„(£) as 
t -> oo. □ 

Remark 1. The choice of the CRM as in (2) essentially makes 
the reference model nonlinear, as x depends on 9 which in turn 
depends on x m in a highly nonlinear manner. In general, the 
tracking problem in a ORM-adaptive system can be viewed 
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as one where an overall nonlinear time-varying system is to 
be designed such that its output tracks that of a linear time- 
invariant system. The CRM-adaptive system is one where 
the overall nonlinear system is instead required to follow a 
nonlinear (reference) model. This nonlinear model, however, 
is chosen such that it asymptotically approaches the original 
linear reference model in the classical case, and hence the 
CRM-adaptive system retains all the desired characteristics of 
the ORM-adaptive system. As we will show in Section [HI] 
the CRM-adaptive system has an additional desirable property, 
of quantifiable transient properties, which the ORM-adaptive 
system may not necessarily possess. We will also show in this 
section that this is made possible by virtue of the additional 
degree of freedom available to the adaptive system in the form 
of the feedback gain in the CRM. 



A. Preliminaries 

All norms unless otherwise noted are the Euclidean-norm 



and the induced Euclidean-norm. The variable t G 



denotes 



time throughout and for a differentiable function x(t), ^x(t) 
is equivalent to i(t). Parameters explicit time dependence (t) 
is used upon introduction and then omitted thereafter except 
for emphasis. The other norms used in this work are the 
£2 and truncated £2 norm defined below. Given a vector 
v G R" and finite p G N >0 , \W(t)\\ Lp = ( J™\W(s)\\Pd S ) 1/p , 

\\v(t)\\L p . T = (/o T |l !/ ( s )ll P ^' s ) 1 ^ anc ' tne infinity norm is then 
defined as H^OHl^ = sup||f(t)||. 



Definition 1. Given a Hurwtiz matrix A m G 1 

a = -max(real(Ai(^4 m ))) 

i 

\\{A m + Al)/2) 

I A„||. 



s = — mm I 



(12) 



For ease of exposition, throughout the paper, we choose L in 
(Hi and Y in C[7]> as follows: 



T — -PT 



(13) 
(14) 



Lemma 4. The constants a and s are strictly positive and 
satisfy 

s>a>0. 

Proof: A m is Hurwitz and therefore a > 0. It is not 
necessary however that the sum A m + A^ is Hurwitz. The 
trace operator is denoted as Tr(-) and is a linear operator. 
Recalling that a matrix and its transpose have the same trace 
we can conclude that J^iXi(A m ) = Y,i\(A m + A 1 ^) / 2. Finally 
we have that 



1 



n 

E 

i=l 



1 

-VA,(i„.) > a > 0. □ 
n — ' 



Lemma 5. With L chosen as in (113b , A m Hurwitz with 
constants a and a as defined in (112b . P in (Q satisfies 



(i) 
(ii) 



IPII <■ 



minA^P) > 



cr + 2£ 

1 



(15) 
(16) 



2{s + £) 

where m = (1 + 4>f)"~ 1 and x = ^. 
Proof: See Appendix iBl 

B. Projection Algorithm 

Before we evaluate the benefits of closed-loop reference 
models, we introduce a modification in the adaptive law to 
ensure robustness properties. 

Assumption 2. A known 8* nax exists such that < ^ ax . 

The projection based adaptive law, which replaces (O, is 
given by 

6(t) = Proj r (6(t),-xe T Pb,f) (17) 

where the T-projection function, Proj r , is defined as in 
Appendix |A] and / is a convex function given by 



\0\\ 



if 



2ed - e 2 



(18) 



where $ and e are positive constants chosen as 1} = 0^, ax and 

£ > 0. 

Definition 2. Using the design parameters of the convex 
function /(#;#, e) we introduce the following definitions 



Otwx = & + £ and 

0max = 2tf + £. 



(19) 



C. Convergence of the Adaptive System 

Theorem 6. Let Assumptions [7] and [2] hold. Consider the 
adaptive system defined by the plant in ([T]l with the reference 
model in (O, the controller in (O, the adaptive tuning law in 
(117t and L and T as in ( 1 1 3b - d 14b - For any initial condition 
in e(0) G R", and 6(0) such that \\6(0)\\ < max , e(t) 
and 9(t) are uniformly bounded for all t > and converge 
exponentially to a set £ through V in (O as 



V < —OL\V + OL2 



where 



and 



with 



2t 



and a 2 = max , 



(20) 



(21) 



{(e,0)||| e || 2 </3i^, ||0|| <9 ma , } 



(22) 



Proof: See Appendix ICl 
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D. CRM free design parameters 

In this section it is argued that the free design parameters 
for the adaptive system are I and the ratio of I and 7. It is clear 
from < [20b that the rate of exponential decay of the Lyapunov 
function is solely a function of the slowest eigenvalue of a and 
I. The term a is defined by the open-loop reference model 
jacobiam A m in © and cannot be independently increased 
without also increasing the bound on 6*, due to the fact 
that A m = A p + b0* T from ©. Therfore, from (ED it is 
clear that while increasing a increases a.\ it also increases 
the size of of the compact set £ that the model following 
error is exponentially converging to. The Luenberger gain I, 
in contrast, does not affect the matching condition for the 
adaptive system and therefore increasing £ does not result in a 
high-gain matching condition. It is seen from (l22l that while 
increasing I may result in a larger a.\ which is desirable, it also 
increases the a 2 and /?i . At first glance this seems undesirable. 
However, it is important to note that £ is inversely proportional 
to the bounds for P in ( TT~5T > and ( fT6] l. Upon further inspection 
of the tuning law in (fTTI i. when Projeciton is not active the 
adaptive law reduces to 

6 = -jxe T Pb, 

and therefore, the only fixed parameters that control the rate of 
adaptation are 7, P(£) and b. Given that b is fixed it is ignored 
and thus the learning rate is a function of 7||P||. Therefore, 
if 7 and £ are increased at the same rate, the effective rate 
of change in the adaptive tuning law will remain the same, 
which illustrates that the ratio of £ and 7 is an important design 
parameter. We therefore introduce p, an effective learning rate, 
as 



P = 



7 



(23) 



Using the definition of p the bounds from Theorem [6] can 
be rewritten as 



a + 21 A 
a 1 = — and a 2 = 



2 ,0 



Ih 



2s 1 
a p 



9 max 1 



(24) 



This reparameterization in terms of p and £ is used for 
discussing the transient performance. 

III. Transient Performance of CRM-adaptive 

SYSTEMS 

In the following subsections we derive the transient proper- 
ties of the CRM-adaptive systems. Five different subsections 
are presented, the first of which quantifies the Euclidean and 
the ^2-norm of the tracking error e. In the second subsection 
we compute the same norms for the parameter derivative 6{t). 
In both cases, we show that the £2-norms can be decreased by 
increasing I. In the third theorem, we address the performance 
of the true error e° and show its dependence on I. In the fourth 
subsection, we define our metric for transient performance in 
terms of a truncated £ 2 norm of the rate of control effort. The 
last subsection compares ORM and CRM adaptive systems 
using these metrics. 



A. Bound on e(t) 

Theorem 7. Let Assumptions Q] and \2\ hold. Consider the 
adaptive system defined by the plant in {T} with the reference 
model in ©, the controller in (|5j, the adaptive tuning law in 
(117b and L and T as in ( I13l l and dl4t . 

21 



||e(t)r< rtl ||e(0)f exp(-- 
1 



|e(*)|| 



T < ~ 

o~ -\- £ 



m||e(0)|| 



7' 



t + 



0(o)|| 



«2 2 2 



where n iy i = 1,2 are independent of p and 
Proof: see Appendix ID] 



(25) 
(26) 

□ 



B. Bound on 9(t) 

In addition to ||e(t)||z, 2 we explicitly compute upper bounds 
for ||0(t)|| and \\0(t)\\ L2 . From the definition of 6(t) in JT71> . 
it follows that 



l|0(*)ll < lir|| 

We note that x(t) = e(t) H 
IkrnC*)!! <x m (0)mexp 



\P\\\\b\\\\x(t)\\\\e(t)\\. 

x m {t) and from © and ([89]l that 



m / exp 1 
'0 



■-(t~r)) (*||e|| + ||6||||r||)dT 

(27) 

Using the bound for ||e(t)||i, 2 from ( fT71 ) and the Cauchy- 
Schwartz inequality, we simplify d27| i as 



lkm(*)|| < x m (0)mexp (-ft) + 



£m , 



e(t)\\ 



r 2||6||m 

a 
(28) 

The above bounds make the following theorem possible. 

Theorem 8. Let Assumptions \J\ and [2] hold. Consider the 
adaptive system defined by the plant in (03 with the reference 
model in (0, the controller in (O, the adaptive tuning law in 
( 1171 ) and L and T as in (TBI) and (fl4l . 



\\9(t)\\ < p exp(-^i) \ax + Vl 

+ Vp cx p (-§ t) a 4 + J - p 

+ \JTpae + \Tiai + pa s 
bi 



a-i + 0,3 



exp 1 



W)\\L <p 2 "(p) 



\ZHpjb 2 



63 



(30) 



where v{p) — m||e(0)|| 2 - 
independent of p and £. 

Proof: see Appendix! 



ij|(9(0)|| , and the a% and bi are 



□ 



C. Bound on e°(t) 

Here, we derive a bound on the true error e°(t) defined in 
O. 

Theorem 9. Let the assumptions from Theorem \E\ hold. The 
difference between the open-loop reference model and the 
closed loop reference model satisfy the following bound 



\\e°(t)\\ <\\e(t)\\ + \ -m\ffp) 



(31) 
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Proof: see Appendix iGl 



□ 



D. Bound on u(t) 

We now derive a final transient measure of the CRM- 
adaptive system that pertains to u. This is chosen as the 
transient performance metric because the rate of change of the 
control authority requested by the controller directly affects the 
robustness of the system to unmodelled dynamics and actuator 
rate limits. Before the bounds are derived, several variables 
must be defined. 



Definition 3. Let time-constants T\(£), t 2 be defined as 

m 2m2 a 2 
a + 2£ a 

Let constants ag and Si(£,N) be defined as 

ag =a + ||6||0 max , 
6 1 (£,N)=cxp (a e N Tl (£))-l. 

where N > 0, and three intervals of time 

Ti = [0,Nn) 
T 2 = [Nt u Ti) 
T 3 = [Ti.oo) 



(32) 



(33) 



(34) 



where T x = max{iVY 2 ,T(e, -£T nX „)} and T(e,-£I nXn ) is 
defined in Corollary [2] 

Remark 2. t\(£) is a time constant associated with the 
exponential decay of ||e(f)|| which is derived from the upper 
bound on V from d20b and t 2 is the time constant associated 
with A m in (|2). ag is a positive scalar that upper bounds 
the open-loop eigen values of A. p from (f2]i and Si(£) will 
be used in the following Lemma to formally define our time 
scale separation condition. The time interval Ti is the time 
interval over which ||e(f)|| decays by N time constants, T3 is 
the asymptotic time scale for e(t) and T2 is an intermediate 
time interval. We note that Ti exists but is unknown. 

Lemma 10. For any N > an £* exists such that 
(i) 6i(£*,N) < 8 where < S < 1. 

(ti) n(£*) < t 2 . 

Remark 3. The condition LemmafTOl(i) defines the time scale 
separation condition. Recall that n is the time scale associated 
with e(t) and ag is an upper bound on the uncertain open-loop 
eigen values of the plant. When I > £* we are able to show 
that at <jv = Ntx, e(t/v) has exponentially decade by N time 
constants, while x(t^) has not deviated far from x(0). 

Assumption 3. 3r ,ri > s.t. \r(t)\ < r , \f(t)\ < r\. 

Remark 4. The bound on f(t) is needed so that u(t) is well 
defined. The analysis techniques that follow in proving Theo- 
rem[TT]will still hold for reference inputs with discontinuities. 
The metric for transient performance however would change 

• T (t)x(t)). 



from u to 4r 



We note that the same analysis holds for x m (0) with addi- 
tion of exponentially decaying terms proportional to x m (0). 

Theorem 11. Let Assumptions l^t- hold. Given arbitrary ini- 
tial conditions in x(0) € l n and ||#(0)|| < 9 max , for any e > 0, 
N > and £ > £* , u satisfies the following inequalities: 



teTi 



for i = 1,2,3, where 



t)\<^-L\\b\\Ge^Gl 



o + 2V 
+ Omax {aeG Xii + ro) + n 



(35) 



Gx,i=(l + 4)||e(0)|| + ^r 



G e ,i =V^T||e(0)|| 



(36) 



G x . 2 ±K 3 \\e(0)\\ + (l + K 4 t) xl 

P 

G e ,2 =V^l|l e (°)ll £ l + \ —9max 
V P 

G x . 3 =K6||e(0)|| + e + (1 + nd) s f^e max + K5 r 

V P 

G e , 3 =e 

where e\ = cxp(-N) and the m are independent of p and £, 
and N > 3 

Proof: see Appendix IH1 

Remark 5. There are two "small" terms in the above analysis. 
61 is determined by the number of time constants of interest, 
e is free to choose and from Corollary [2] proves the existence 
of a finite T and is used to define when T3 begins. 

From Theorem [TT] it follows that 

sup \u(t)\ <cip + c 2y r p + ri 
teii 

sup \ii(t)\ <4~pc z + (1 + al)c 5 + J-(l + c 4 £) 2 ce 

t(ET 2 V P 

+ e 1 £ 1 (p,£,^,£^,£ 2 )+r 1 ( 37 ) 
1 



Assumption 4. For ease of exposition we will assume that 

&m(0) = 0. 



sup \u(t)\ <\ -(1 + c 4 £)c 7 + c 8 

tST 3 V P 

+ e£ 2 (p,£,y/p,£y/p, £ 2 ,e 1 ) + n 

where Ci > 0, i = 1 to 8 are independent of £ and p, £i(-) 
and £2(0 are globally lipschitz with respect to their arguments. 
The inequalities in ( f37T > lead us to the following three main 
observations (see Figure Q]) 

(Al) Over Ti, \u(t) \ is bounded by a linear function of p and 

Vp> 

(A2) Over T2, \u(t)\ is bounded by a linear function of 

(A3) Over T3, \u(t)\ is bounded by a linear function of 
and £ 

(A4) Ti decreases with 

Remark 6. The main idea used for the derivation of the 
bounds in Theorem [TT] is time-scale separation of the error 
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dynamics decay, and the worst case open-loop eigenvalues of 
the uncertain plant. The most important point to note is that n 
can be made small by choosing a large I. There is a penalty, 
however, in choosing a large I, as the bound G x , 2 increases 
linearly with i. Therefore, after choosing an £ which satisfies 
the time scale separation as needed in Lemma [10] a p (which 
through d23l defines a choice for 7) can be chosen such that 
the integral in the following theorem is minimized. 



Theorem 12. There exist optimal p and i such that 



(A 



optAopt) = argmin||u(p, £)\\ 



L 2 ,r 



(38) 



for any < r < T\. 



Proof: ||u(/9,£)||l 2jT is continuous with respect to p and I 
where p and £ appear in the numerator of d37l i and are positive. 
Therefore, p opt and £ ovt exist and are finite. □ 
t in Theorem [12] denotes the interval of interest in the 
adaptive system where the transient response is to be con- 
tained. Given that T, and therefore Ti is a function of £, 
d38l can only be minimized over Ti U T2. From the authors 
definition of smooth transient performance in the beginning 
of this section choosing p opt and £ opt will guarantee smooth 
transient performance. 

E. Comparison of CRM and ORM-adaptive systems 

The bounds on e(t) and the £2~norm of 8 directly show 
that CRM-adaptive systems lead to smaller e(£) than with 
the ORM which are obtained by setting I = in d25b and 
( l26l >. However, the same cannot be said for either e° or for 
the Euclidean norm of 8; for a non-zero I, the bound on e° 
is larger than that of e. This indicates that there is a trade- 
off between fast transients and true tracking error. The signal 
that succinctly captures this trade off is it, whose behavior is 
captured in detail using the time intervals Tl, T2, and T3. We 
also showed in Theorem[T2]that this trade-off can be optimized 
via a suitable choice of £ and p. In what follows, we compare 
this optimized CRM with ORM and show that the former is 
clearly better than the latter. 

Definition 4. The following two time constants 



r^n(O) 



2m 2 



and Ti = ri(f ) 



(39) 



are used to describe the three time intervals that will be used 
in the analysis of u for the ORM case 

Ti = [0,Nt*) 

T' 2 =[Nt 2 ,T{) (40) 
T 3 = [Ti',oo). 

where T[ = max{NT 2 ,T(e, 0)} where T(e, 0) is from Corol- 
lary m 

As in Definition [3] here too, T exists but is unknown. While 
these periods for both CRM and ORM are indicated in Figure 
[TJ one cannot apriori conclude if Xi is greater than or smaller 
than T[. The time instants indicated as in Figure [TJ are meant 
to be merely sketches. 




Fig. 1. Transient bounds for it. 



Proposition 13. Let 



po = 



(41) 



For the adaptive system with the classical MRAC given by 
Eqs (D, ©, ©, CCD"® and ©-CK with £ = 0, it can 
be shown that 

sup \ii{t)\ <p di + y/p^d 2 + ri, 

T 



sup \u(t)\ ^yfpa'dz + d± + J— d 5 + ei97ti(p , Vpo) + r i 
ten, V Po 



sup \u(t)\ <J — d 6 + d 7 + effl 2 (p , VPo) + r l 

t£T 3 V Po 

(42) 

di > 0, i = 1 to 7 are independent of po, and SEJti(-) and 
9Jl2(-) are globally lipschitz with respect to their arguments 

The proof of Proposition [13] follows the same steps as in 
the proof of Theorem QT] and is therefore omitted. 

The bounds in d42l indicate that in the classical ORM, one 
can only derive a bound for u over the period Ti, T 2 and T3. 
Unlike the CRM case, the procedure in Appendix [H] cannot 
be used to derive satisfactory bounds for u over [iVY* , iVr^ ) . 
It also can be seen that unlike the CRM case, t' 2 is fixed and 
cannot be changed with £. These points are summarized below. 
(Bl) Over T[, \ii(t)\ is bounded by a linear function of po 
and yfpE 

(B2) Over T 2 , \u(t)\ is bounded by a linear function of ^/po 

and * / — 

V Po 

(B3) Over T3, 1^(^)1 is bounded by a linear function of 



(B4) t' 2 is fixed and unlike ri, can not be adjusted. 

We now compare the bounds on u using observations (Al)- 
(A3) and (B1)-(B3). In order to have the same basis for 
comparison, we assume that 7, a, and £ are such that p = p 
and that both CRM- and ORM-adaptive systems start with 
the same bound at t = 0. As noted above, a tight bound 
cannot be derived for the ORM-based adaptive system over 
[Nti , Nt 2 ). In the best scenario, one can assume that this 
bound is no larger than that over [0, Nt*]. This allows us to 
derive the bounds shown in Figure [TJ The main observations 
that one can make from this figure are summarized below: 

. Even though at time t = 0, both the ORM and CRM 
have the same bound, since n can be made much smaller 
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Fig. 2. Plot of |ii(f)|. 

than t' 2 , this bound is valid for a much shorter time with 
the CRM-system than in the ORM-system. This helps us 
conclude that the initial transients can be made to subside 
much faster in the former case than the latter, by suitably 
choosing I. 

• The bound on ii for T2 with the CRM-adaptive system 
is however linear in powers of I and hence can be larger 
than the bound on u with the ORM-adaptive system over 
T' 2 . 

• The above observations clearly illustrate, if the cost func- 
tion U (NT2, p., £) is minimized then the CRM system will 
have smoother transients than the ORM. Then, at larger 
times the error dynamics will asymptotically converges 
to zero. 

F Water-Bed Effect 

The discussions in the preceding sections clearly show that 
CRM-adaptive systems introduce a trade-off: a fast conver- 
gence in e(t) with a reduced ||#(£)||l 2 occurs at the expense 
of an increased e°(t). While an optimal choice of p and 
I can minimize this trade-off, it also implies that a badly 
chosen I and p can significantly worsen the adaptive system 
performance in terms of e°(i) and «(£). We denote this as 
the water-bed effect and illustrate it through a simulation. 
Consider a first-order plant with a single unknown parameter, 
whose values are identical to the example in Section lTV-Al over 
the first ten seconds. Figure [2] shows the behavior of ii(t) for 
the ORM, the optimized CRM, and a poorly chosen CRM. 
The plots clearly show the water-bed effect for the last case 
and the improved performance of the optimized CRM over 
the ORM. The free design parameters are also shown in the 
figure. 

IV. Robustness of CRM to Time- Varying 
Uncertainties and Disturbances 
We now evaluate the CRM-adaptive system in the presence 
of perturbations due to time-varying parameters and distur- 
bances. Consider the uncertain Linear Time Varying system 



x = A p (t)x(t) +bu + d(t) 



(43) 



where d(t) is a bounded disturbance and A p (t) is time varying 
with a bounded time-derivative. It is assumed that a time- 
varying vector 0*(t) exists such that 

A m = A p (t) + b9* T {t), (44) 



and 9% 9 max exist where \\8*(t)\\ < 8* and \\9*{t)\\ < 9 max . 

Theorem 14. With Asswnptions\l\and\2\ consider the adaptive 
system defined by the plant in d43l ) with the reference model in 
(O, the controller in ©, the adaptive tuning law in d 1 7b and L 
and r as in d!3l )- (ll4l) . For any initial condition in e(0) £ K™, 
and 8(0) such that ||#(0)|| < 9 max , e(i) and 9(t) are uniformly 
bounded for all t > and the Lyapunov candidate in (O 
converges exponentially to a set £ as 



V < — a^V + «4 



where a% = ^f, 



0:4 
and 

£ = {(e,9) 
where 



<t + 2£~ 2 2 - 

~7i 9 inax ~l d d 9 max + 2 

2to z 7 7 



(7 + 21 



(45) 



\d(t)\\ 2 , (46) 



|e|| 2 < PiOLc + MlOnua + W\\ 2 , \\9\\ < ~9 max ) 



A 8sm 2 

P2 = and p 3 

(77 



Asm 



0(0 + 



(47) 



Proof: See Appendix [F] 



Remark 7. From the above Theorem it is is shown that in 
the presence of disturbance and time-varying uncertainty, the 
rate of exponential decay of the lyapunov function is ai/2 
where a\ was the rate of exponential decay for the system 
with constant uncertainty and no disturbances. The term that 
controls how the compact set £ grows with the upper bound 
on the rate of change of the time-varying matched uncertainty 
is /?2 which is inversely proportional to 7. (3 3 scales the size of 
the compact set in regard to the disturbance term d. Therefore, 
increasing 7 and I in proportion so that p is constant constant 
decreases the size of the compact set £. 

A. Simulation Study 

For this study a scalar time varying system of the form in 
(l43l is controlled where 

{1 < t < 20 

l + i(i-20) 20 <t<24, 
2 t > 24 

6=1, and d(t) is a deterministic signal used to represent 
a disturbance. Over the first 20 seconds d(t) = 0. After 20 
seconds d(t) is generated from a Gausian distribution centered 
at with a variance of 1, co variance of 0.1, deterministically 
sampled at 10 Hz with a fixed seed, and then passed through a 
saturation function with upper and lowers bounds of 0.2 and - 
0.2 respectively. The reference model to be followed is defined 
as 

x m = -x m + r + i[x - x m ) (48) 

with control input from (O and the update law for the adaptive 
parameter defined ©, where I and p are chosen as in Table 

m 

The simulations have three distinct regions of interest, with 
Region 1 denoting the first 10 seconds, Region 2 denoting the 
10 sec to 20 sec range, and Region 3 denoting the 20 sec to 
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Region 1 Region 2 Region 3 

Fig. 3. (top) reference model trajectories x m , (middle) state x, and (bottom) 
model following e. 




Region 1 Region 2 Region 3 

Fig. 4. (top) Control input u, (middle) rate of control input it, and (bottom) 
adaptive parameter 9(t). 

TABLE I 

Test case free design parameters 

Parameter Open-Loop Closed-Loop 
1 To 
p 100 100 



35 sec range. In Region 1, the adaptive system is subjected to 
non-zero initial conditions in the state and the reference input 



is zero. At t = 10 sec, the beginning of Region 2, a filtered 
step input is introduced. At t = 20 sec In Region 3, time- 
variations in the plant parameter as well, disturbances and a 
filtered step input are introduced. Figures 2 and 3 illustrate the 
response of the CRM-adaptive system over to 35 seconds, 
with x m , x, and e indicated in Figure 2, and u, u, and 9 
indicated in Figure 3. In both cases, the resulting performance 
is compared with the classical adaptive system. The first point 
that should be noted is a satisfactory behavior in the steady- 
state of the CRM-adaptive controller. In particular, as can be 
seen from the latter half of Region 1, both e and e° tend to 
zero as t approaches 10 seconds. The same satisfactory trends 
are observed in regions 2 and 3 as well, underscoring the 
robustness property of CRM-adaptive control, which validates 
Theorem 2. 

We also note yet another significant difference between 
the responses of CRM-adaptive control and the classical one, 
which pertains to the rate of control input u. An examination 
of Regions 2 and 3 clearly illustrates that the control input is 
smoother for CRM-adaptive control. 

V. CMRAC 

We now return to CMRAC introduced in [Q7) and (T8). We 
will show that the introduction of a CRM in these adaptive 
systems not only ensures stability, but also enables stability 
with observer-based rather than state-based feedback. In ad- 
dition, the use of a CRM in CMRAC enables the derivation of 
transient properties which could not be accomplished hitherto. 
Section A addresses stability of the CMRAC with CRM, 
denoted as CMRAC-C, in the scalar case. Section B extends 
the results from section A to higher order plants with states 
accessible and addresses transient properties of the CMRAC-C 
adaptive system. Section C introduces an additional feature of 
observer feedback. Denoting the underlying adaptive system 
as CMRAC-CO, it is shown that the resulting adaptive system 
has guaranteed stability properties and results in reduced error 
bounds in the presence of measurement disturbances with a 
zero mean property, which is corroborated through simulations 
in Section D. Section E contains extensions to higher order 
plants whose states are accessible. 

A. Stability of CMRAC-C 

We assume that the plant and reference model dynamics 
are given by Equations (Q~|i and (f2]i with A m and L = L m 
satisfying Equations (0|i and ^3), The control input is chosen 
as in d5j and the identifier dynamics are given by 

±i(t) = Lifait) - x(t)) + (A m - b§ T {t))x(t) + bu(t) (49) 

where Li is Hurwitz. The error dynamics are now given by 

e m (t) =(A m + L rn )e m + W T {t)x 
ei(t) =Liei — W T (t)x, ei = xi — x 

where 8(t) = 6{t) — 9* . For ease of exposition we choose 

L m = L = -Unxn and L t = -(a + £)I nxn - (51) 
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(52) 



The update laws for the adaptive parameters are then defined 
with the update law 

9 = Prqj r (0(i), -xe[ n P m b, f) - 7]I n x n ee 

6 = Pmj T {0{t),xe'[P l b, f) + T]I nxn e e 

where eg = 9(t) — 9(t), with V chosen as in ( fT4t . rj > 0, and 
P m = P from O and Pi = 2(g 1 +£) /nxK- 

Theorem 15. Let Assumptions [JJ and \2\ hold. Consider the 
overall CMRAC-C specified by CD, ©, ©, gill, d23 ^ 
( 152l >. For a«y initial condition e m (0),ei(0) € R", and 9(0) 
and 9(0) such that ||#(0)|| < 9 max and ||#(0)|| < 9 max , it can be 
shown that e m (t), e (t), 9(t) and 9(t) are uniformly bounded 
for all t > and the function 

~9 T T' 1 9 + 9 T T- 1 9 (53) 



converges exponentially to a set £ as 

V < -aiV + 2a 2 

where 



(54) 



{« 



||e m f < ^4^, lleiH 2 < fol 
\\0\\ <9 max , \\9\\ <9 max } 



|2 

max 



with 



A 4(.s + I) A 4(a 
P4 = ana ps = 



(55) 



Proof: see Appendix [J] 



Remark 8. There is no appreciable difference between the 
CMRAC-C and CRM adaptive controller presented in Section 
|TT] in terms of stability and the bounds for the set £. 

VI. Transient Properties of CMRAC-C 

In the following subsections we derive the transient prop- 
erties of the CMRAC-C adaptive system, similar to what was 
done in Section [Til] Two different subsections are presented, 
the first of which quantifies the Euclidean and the £ 2 -norm 
of the tracking error e and the second subsection, were the 
truncated £ 2 norm of the rate of control effort is presented. 

A. Bound on e m (t) and ei(t) 

Theorem 16. Let Assumptions [JJ and [2] hold. Consider the 
overall CMRAC-C specified by ©, ©, ©, ([49), (E3 and 
(152l >. For any initial condition e m (0),ei(0) € R n , and 9(0) 
and 9(0) such that \\9(0)\\ < 9 max and ||0(O)|| < 9 max . 

Wm{t)f <k 7 (l|e m (0)|| 2 + ||e. t (0)|| 2 ) expi-att) 

+ Ilfj 2 (56) 

' max 

lle.Wf <||e m (t)|| 2 (57) 
1 



l|e m (*)|| 



<- 



-<j + £ 



(m 2 ||e m (0)|| 2 + |M0)|| 2 ) 



1 



-\\m\\ 2 
p 



1 0(o)|| 



\\ei(t)\\* <\\e m (t)\\l 



where Ki, i = 7, 8 are independent of p and £. 
Proof: see Appendix iKl 



(58) 



(59) 



□ 



B. Bound on u(t) 

Definition 5. The following three time intervals are used when 
exploring the transients of CMRAC-C 

T'( = [0,N n ) 

T'> = {NnX') (60) 
T' 3 ' = [Tf,oo) 

where T" = mzx{NT 2) T(e,-£I nXn )}, with T(e,-£I nXn ) 
following from the application of Barbalat Lemma to the 
adaptive system defined in Thereom[l5]for any e > (identical 
to Corollary [2). 

Theorem 17. Let Assumptions 1-4 hold. Given arbitrary 
initial conditions in x(0) £ W 1 and \\9(0)\\ < 9 max , if I > £' 
the derivative u satisfies the following two inequalities: 



sup |ix(t) I < 

t€T." 



m 2 7 
7 + 2V 



\G'e,iG",i + 8r]9f nax G" 



(61) 



(a e G xi + r Q ) + n 



where 
G'x.i =( 

u e,l — 



+ *i)||e(0)| 
«7 (||e m (0)|| 



G" x ,2 =«9 (11^(0)11 + 11^(0)11) 

+ «llTo 



G':^^(\\e m (0)\ 



T e,2 

G% ^ 12 (||e m (0)| 
+ (2 + K W £) 




(62) 




Umax + Kll^O 



with ei = cxp(— N) 



Proof: The finite time stability result used in ( 1107b still 
holds for the MMRAC-C. Therefore G xl in ([62) is identical 
to G Xt i in (f36b . The Lyapunov function in d53l has two 
additional terms in and as compared to the Lyapunov 
equation in (|9). Therefore, Gg% now includes the initial 
conditions of the estimation error ej(0). G" 2 and G" 2 are 
similarly affected. Barbalat Lemma can be used for G" 3 , and 
G" 3 follows from the same analysis in Appendix IH-CI The rj 
terms arise from the righthand side of the update law in ( 1521 . 

□ 



VII. CMRAC-CO 

When measurement noise is present, it is often useful to 
use a state observer for feedback rather than the plant state. 
However, the use of such an observer in adaptive systems has 
proved to be quite difficult due to the inapplicability of the 
separation principle. In this section, we show how the CRM 
can be used to avoid this difficulty for a class of plants. We 
denote the resulting adaptive system as CMRAC-CO. 
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We assume that the plant and reference model dynamics 
are given by Equations (fl]i and (O with A m and L = L m 
satisfying Equations and ([3]). The control input is now 
chosen as 

u = 9 T (t)x Q + r (63) 
and x is the state of the observer dynamics, given by 

x Q (t) = L (x (t)-x(t)) + (A rn -b9 T (t))x (t) + bu(t). (64) 

Defining e m (t) = x(t) — x m (t) and e (t) = x a — x(t), the 
error dynamics are now given by 

e m (t) =(A m + L m )e m + b9 T (t)x Q + b9*e a 



e (t) =(A m + L - b6*)e - b9 T (t)x . 
For ease of exposition we choose 

L m = L a = L = —ilnxn- 



(65) 



(66) 



The update laws for the adaptive parameters are then defined 
with the update law 



9 = Proj r (6>(t), -x Q ef n Pb, f) - ?y/„xn£e 
9 = Pmj r (9{t),x e^PbJ) + V I n 
with T chosen as in ( TBi i, r\ > 0, with P from ©. 
Lemma 18. Let 



(67) 



A(£) 



A 4m 2 ||6|| 



(68) 



a + 21 

Then, there exists an I" such that < A(£") < 1. 

Theorem 19. Let Assumptions [JJ and |2] hold with I chosen 
such that I > £". Consider the overall CMRAC-CO specified 
by (Q3, ©, (163l l, (I64l l, ( 165b and (167) . For any initial condition 
e m (0),e o (0) £ R n , a«t/ 6»(0) a«<i 9(0) such that \\9(0)\\ < 
9 m ax and ||0(O)|| < 9 max , it can be shown that e m (t), e (t), 
9(t) and 9(t) are uniformly bounded for all t > and the 
trajectories in the function 

V = e T m P m e m + e T P e + T r _1 + (69) 

converge exponentially to a set £ as 

V<-a 5 V + a 6 (70) 

where 

(1- A(l))(a + 2l) 

(71) 



«5 



a 6 



,2(l-A(f))( ff + 2l) fl - ; 



7W 2 



2 

IIHIA 



and 



{< 



with 



!|e m !| 2 < /^e^Lxj ll e o|| 2 < Ad^Lx 
ll^ll <^«, Pll <^, a ,j 

4(s + 1) 



1 



Proof: see Appendix [E] 



(72) 
□ 



A. Robustness of CMRAC-CO to Noise 

As mentioned earlier, the benefits of the CMRAC-CO is 
the use of the observer state x Q rather than the actual plant 
state x. Suppose that the actual plant dynamics is modified 
from (Q]i as 

x a (t) =A p x a (t)+bu{t), x{t) = x a {t) + n(t) (73) 

where n(t) represents measurement noise. For ease of expo- 
sition, we assume that n(t) is bounded and deterministic. 
This leads to a set of modified error equations 

e m (t) ={A m + L m )e m + b9 T (t)x D + b9*e D + L m n(t) 
e Q {t) =(A m + L - b9*)e - b9 T (t)x - L n(t) 

Theorem 20. Let Assumptions |7J and |2] hold with I chosen 
such that I > £". Consider the overall CMRAC-CO spec- 
ified by CD, ©, d63j, (O, dZ4]l and ([67). For any initial 
condition e m (0),e o (0) £ M™, and 9(0) and 9(0) such that 
||^(0)|| < 9 max and \\9(0)\\ < 9 max , it can be shown that e m (t), 
e (t), 9(t) and 9(t) are uniformly bounded for all t > 
and the trajectories in the function V from ( |69t converges 
exponentially as 

V < -a 7 V + a 8 (75) 



(74) 



where 



A (l-A(£))(a + 2£) 
a 7 = — 



OS 



2m 2 

4 (l-A(l))(<r + 2l) - 2 

7 TO 2 

16 



(76) 



(1- A(£)Y \cr + 



21 



\n(t)f 



and 



£ 4 {(e m ,e o ,0>")|||e m || 2 < (5 e 9f nax + f5 r \\n(t)\\ 2 , 
\\e \\ 2 <Ml ax + (3 7 \\n(t)\\ 2 , 



< 



9 max ; 1 1 9 1 ^ 9 max } 

with (3q defined in ( 1721 ) and /3 7 defined as 

A 64m 2 s 
17 = <t(1-A(£)) 3 
Proof: see Appendix IM1 



(77) 



B. Simulation Study 

For this study a scalar system in the presence of noise is 
to be controlled with dynamics as presented in d73l ). where 
A p = 1, b = 1, and n(t) is a deterministic signal used to 
represent sensor noise. n(t) is generated from a Gausian dis- 
tribution with variance 1 and co variance 0.01, deterministically 
sampled using a fixed seed at 100 Hz, and then passed through 
a saturation function with upper and lower bounds of 0. 1 and 
-0.1 respectively. The reference model, identifier and observer 
are from (O, d49l and d64l ) respectively, with A m = — 1 and 
b = 1. The controller is defined by d67l ). The design parameters 
for the two test cases are shown in Table [ill 

The simulations have two distinct regions of interest, with 
Region 1 denoting the first 4 seconds, Region 2 denoting the 
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open-loop 
closed-loop 




Fig. 5. (top) reference model trajectories x m , (middle) state x, and (bottom) 
model following e. 

TABLE II 

Test case free design parameters 
Paramater Open-Loop Closed-Loop 
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Fig. 6. (top) Control input u, (middle-top) discrete rate of change of 
control input Au/At, (middle-bottom) adaptive parameter 6(t) and (bottom) 
adaptive parameter 9(t). 



4 sec to 15 sec range. In Region 1, the adaptive system is 
subjected to non-zero initial conditions in the state and the 
reference input is zero. At t = 4 sec, the beginning of Region 
2, a filtered step input is introduced. Figures [5] and [6] illustrate 
the response of the CMRAC-CO adaptive system over to 
15 seconds, with x m , x, and e m indicated in Figure |5j and u, 
u, 8 and 6 indicated in Figure [6] In both cases, the resulting 
performance is compared with the classical CMRAC system. 
The first point that should be noted is a satisfactory behavior 
in the steady-state of the CMRAC-CO adaptive controller. 
We note a significant difference between the responses of 
CMRAC-CO and CMRAC systems, which pertains to the 
use of noise free regressors in CMRAC-CO. An examination 
of Au/At in Figure [6] clearly illustrates the advantage of 
CMRAC-CO. 

VIII. Comments on CMRAC, CMRAC-C and 
CMRAC-CO 

As discussed in the Introduction, combining indirect and 
direct adaptive control has always been observed to produce 
desirable transient response in adaptive control. While the 
above analysis does not directly support the observed transient 
improvements with CMRAC, we provide a few speculations 
below: The free design parameter Li in the identifier is 
typically chosen to have eigenvalues faster than the plant that is 



being controlled. Therefore the identification model following 
error e.; converges rapidly and 9(t) will have smooth transients. 
It can be argued that the desirable transient properties of the 
identifier pass on to the direct component through the tuning 
law, and in particular eg. 

The CMRAC-C differs from classical CMRAC only due 
to the Luenberger gain L rn in the reference model. Given the 
contributions of Section [III] which show that the CRM can 
result in satisfactory transients without the indirect component 
raises the question if the added complexity of a CMRAC-C 
is justified. One answer to this question is in the form of the 
CMRAC-CO, where it is shown that one can design stable 
observer-based feedback in a CMRAC, allowing noise-free 
estimation and control. 

IX. Conclusion 

This paper concerns the introduction of a feedback gain 
L in the reference model and the analysis of various adap- 
tive systems with this feature. In particular, we show that 
with closed-loop reference models (CRM), (i) direct adaptive 
control structures result in guaranteed transient performance, 
(ii) combined direct and indirect adaptive controllers result 
in guaranteed transient performance, and (iii) observer-based 
feedback can be used in adaptive systems while retaining 
stability. These are primarily realized using the extra degree 
of freedom available in the CRM in terms of a feedback gain, 
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and by exploiting exponential convergence properties of the 
CRM-adaptive system. In all cases, a projection algorithm is 
used in the adaptive law with a known upper bound on the 
unknown parameters. 

The main impact of this work is the quantification of 
transient performance in adaptive systems through £2 norms 
of tracking errors and the control input derivative u. It is 
shown that the introduction of the feedback gain L introduces 
two time-scales to govern the adaptive system dynamics. The 
first has to do with the convergence of the tracking error, and 
the second has to do with adaptation to the unknown plant 
parameter. By allowing these two time-scales to be separate, 
transients in the adaptive systems can be controlled without 
compromising learning of the unknown parameter. This in 
turn is accomplished by choosing L in an optimal manner. 
Sub-optimal choices can result in better transients in e only 
at the expense of slow adaptation leading to a "water-bed" 
effect. This paper, to our knowledge, is the first to illustrate 
this effect via an exhaustive formal and experimental analysis 
of CRM-based adaptive systems. 
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Appendix A 
Projection Operator 

The T-Projection Operator for two vectors 9,y G M fc , a 
convex function f(9) G M and with symmetric positive definite 
tuning gain T G M. kxk is defined as 



Proj r (0,y,/) = 



r „ _ p v/(9)(y/(9)) r r f(&) 

1 y 1 [wf{0)) L rwf(9) L y j y°> 



Jy 



if f{6) > A y T TVf(9) > 
otherwise 



where V f(9) = ( j^g^ ■ ■ • J ■ The projection operator 
was first introduced in ||24|| with extensions in |2j and for a 
detailed analysis of T-projection see |25l . 

Definition 6. The following compact sets will be referred to 
in the following analysis: 

v Q = {9eR k \f(9) <o} 

X>i = {9 G K fe |/(6>) < 1} (79) 
V 5 = {9eR k \f(9)<5}. 

Theorem 21. Given 9 = Proj r (9,y, }), f(9) : M fc ->• M is 
convex, 9* G 2?o an d 9(0) G T>\ 

9(t) G VxMt > and (80) 
(9 - 9*) T (T- 1 Proj r (9,y, f)-y)< 0. (81) 

Before we prove the above theorem, we introduce the 
following two lemmas. 

Lemma 22. Let f(9) : ]R fe — > R be a continuously differen- 
tiable convex function. Choose a constant 5 > 0. Let 9i be an 
interior point ofT>$, defined in (|79t . Choose 9b as a boundary 
point so that /(0f>) = 5. Then the following holds: 



{6i - 9 b y Vf(9 b ) < 

T 



(82) 

where V f(9t) = ' ' ' 9 gg^ ) evaluated at 9^. 

Proof: see ll25l Lemma 4] □ 



13 



Lemma 23. Given 9* G V , 

(9 - 9*) T (T- 1 Proj r (9,y, f) - y) < 0. (83) 
Proof: If f(9) > A y T TVf{9) > 0, then 



and using Lemma l22l 

(0* - e) T v.f{0) 



(V/(0))*TV/(0) 



(v/(8)) T rv/(fl)' 



(v/(0)) T iw(0) < o 



otherwise Proj r (#, y, /) = Ty. □ 
Proof of Theorem 127} We begin by proving (f80b . Con- 
sider the function 

F(0) = /(0) 2 , (84) 

and taking its time derivative 

F(9) = 2f{9){Vf{9)) T 9 

and when f(9) = 1 one has that 

F{6)=2f{6) {Vf{9)) T Vm] r {9,yJ). 

With direct substitution of the operator in ( 1781 one finds that 

(V/(#)) T Proj r (# ;2 /,/) <0 (85) 

whenever f(9) = 1, and thus d80b holds. Equation dSTT i is 
proven with direct application of Lemma [23] □ 

Theorem 24. Given 

6=Pwj T (e, yi ,f)-r,i nxn (e-§), 

d=Pwj r (9,y 2 J) + riI nXn (6-§) 



(86) 



where rj > is a scaler, 9* G X> , 6>(0) £ Pi, 0(0) G X>i and 
/ is convex 



9(t) G 2?iVi > and 
0(i) G T>xVt > 0. 



(87) 



Proof: Given that 9 and 9 both begin in Pi either both 
parameters hit the boundary of T>\ simultaneously or only one 
parameter is at the boundary of T>\ while the other is strictly 
inside. Lets consider the case where 9(t) is on the boundary 
of T>i and thus f(9) = 1 and 9 C T>\, Consider the quadratic 
function F(9) as first presented in (l84l . Differentiating F(9) 
and using the update law in (l86l l we have 



(88) 



F{9) =2f(9) (V/(0)) T Proj r (0,2/,/) 
-nV{0) (Vf(6)) T (6-6). 



From (183b we already know that the first part of d88b is less 
than 0. For the second part, given that f(9) is convex and 
since 9 G £>i, (V/(0)) T (0- 0) > 0, and therefore F(9) < 0. 
The same result holds for F(9), proving (f8Tb . □ 



Appendix B 
Proof of Lemma|5] 

Lemma 25 ( l26l Lemma 1]). Any Hurwitz matrix 
A m G R" xn with constants a and a as defined in (1121) satisfies 
the following bound for the matrix exponential 

||exp(A m r)|| < mg exp((-cr + 6a)r) 



where ms - 
directly from 



1 



and S > 0. The proof follows 



Corollary 26. Setting 5 = a /(2a) the following holds 



||exp(A m r)|| < mcxp \~-^r 
where m = | (1 + 4>r) Tl 1 and k = —. 



(89) 



Lemma 27. For any diagonal matrix L = —ll n xn the follow- 
ing bound holds for the matrix exponential 

||exp(Z/r)|| < exp(-Zr) (90) 

The proof follows from l\27\ Section 2]. 

Proof of Lemma \5%i): Beginning with the 
integral form of Lyapunov's equation in (UJ 
P = J °° exp(A^T) exp(A m T) dr. Due to our choice of L, 
A m and L commute, thus exp(A m + L) = cxp(A m ) exp(L) 
and 

poo 

P= ex P(^m r ) exp(L T r) exp(A m r) exp(Lr) dr. 
Jo 

Using the bound in ( f89t and ( f90b the integral just above can 
be upper bounded and the bound in il5[ directly follows. □ 
Proof of Lemma \5\ ( ii): Let £ G K ra be a normalized 
eigenvector of P. By pre- and postmultiplying (UJ by £ T and 
£, we have 

eA T m Pi + fpAu = -£ t /„ x „£ 

which reduces to 

K{P)(i T {A m + A T m )t = -l. 

Expanding A m we have 

Ai(P)£ T (A m + A T m - 2ZJ nXn )£ = -1. 

Finally, using the definitions in dTzb and taking the minimum 
eigenvalue of P we arrive at ( TTBT ) l28l . □ 

Appendix C 
Proof of Theorem[6] 

Proof: Recall the Lyapunov candidate in ©, Taking its 
time derivative one has that 



V< -\\e\\ 2 < 



1 



-V- 



1 



max 



.\p\\ \\P\h 

Using the upper bound on P from $15[ 

V < —ol\V + a.2 

with ai defined in (ED and a 2 = S^max- 
Gronwall Bellman Inequality, d9Tb implies that 



(91) 
Using the 



V(e, 9) < [ V(e(0), 9(0)) - ^ ) exp(-ait) + — . (92) 

«1 / Q!l 
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Thus, e exponentially converges to the set defined by the 
following inequality 

1« 



lim e(ty Pe{t) < 

t— >oo 



) 2 . 

max 



(93) 



Using the bound in Lemma |3ii) we have that 

then we can conclude that Hindoo ||e(i)|| 2 < /3i0^ ax where 
j3\ is defined in (l22l i. The boundedness of 9(t) follows from 
Theorem I2U □ 

Appendix D 
Proof of Theorem[7] 

Proof: From d92l and j93l , we know that 

||e(t)|| 2 < fc expf-^i^t ) +h 



where 



fc = 



fci 



2(s + f)m 2 

CT + 2^ 

2(8 + ^) ^2 
- y ff ™ 



leCojf + fc^n^o)!! 3 -*! 



(94) 



Using the following inequalities 

2(s + e)m 2 < 2sm 2 ^ 2{s + t) ^ 2s a + 1 



a + 2£ 



and 



< 



a 7 a 7 

the fact that ||0(O)|| < # max and the definition of p from ( l23l . 
the result in (l2~5ll holds with 



2sm 2 2s 
K! = and k 2 = — . 



Beginning with 

/>oo 

||e(t)HL< / -V(e(t),9(t))<V(e(0),9(0)) 
Jo 

<^Mo)f + -\\m\\ 2 , 

a + 21 7 



(95) 



(96) 



using the definitions of p from d23l and the fact that < 
-^— t the bound in d26b holds. □ 



Appendix E 
Proof of Theorem[8] 

Proof: Using ([L5"l l. the choice for Y in (TBI and the 
definition of p from (l23l we have that ||r|| ||P|| < m 2 p. Using 
the bounds in (l28l l and d25l l for ||x m (t)|| and ||e(i)|j the results 
in (|2"9l follow immediately. 

For the £2 norm we begin by observing that 

POO 

||^)||| 2 <||r|| 2 ||P|| 2 ||&|| 2 sup||.T m (t)|| 2 / \\e(t)\\ 2 dt 



+ ||r||lP||i&|| 2 S up||e(*)|| 2 / \\e(t)\\ 2 dt. 

Jo 



Taking the supremum of (128t and (125t we have upper bounds 

for sup||a; m (t)|| 2 and sup||e(i)|j 2 . The £ 2 norm of e(t) is 
given in (126V □ 



Appendix F 
Proof of Theorem[T41 

Proof: Taking the time derivative of the Lyapunov can- 
didate in ©, substitution of the update law from (fTTI i and the 
plant dynamics in (l43l . the derivative of the lyapunov function 
can be upper bounded as 

u<-|| e || 2 + 2!|p||||d||!| e ||- 



7 

After completing the square in e and d we have 

^<-i|| e || 2 -i(j| e ||-2||P||||d||) 2 

+ 2||P|l 2 !|rf!l 2 + ^H0 max 



7 

and then neglecting the negative quantity after — l/2||e|| 2 , 

V < -ke\\ 2 + 2\\P\\ 2 \\d\\ 2 + ^L^. 
2 7 

Writing the above inequality in terms of the Lyapunov candi- 
date in (O we have 



V < 



2 P 



-V 



1 



max 



2||<9* 



2||P|| 2 ||d|| 



(97) 

Using the upper bound on ||P|| from (fTBI l and rewriting ((97} 
in terms of the design parameters 7 and I we have 



V < — (X3V + Q?4 



(98) 



where a-s is defined just before 04 in (l46l i. Following the same 
procedure as in Appendix [C] we conclude that 

™2 x 3 



4m 



211/3*1 



lim e T Pe < —O 2 , , 
t^oo - 7 max (a + 2f) 7 



21 



\d\\ z - 
(99) 

Recalling the fact that s > a > from Lemma [4] we can 
conclude that 



2(s + 1) 2s 



(100) 



a + 21 a 

Using the bound above along with that in d93l the inequality 
in d99l ) can be simplified as 



lim \\e{t)\\ 2 < frOl,, + f3 2 \\e* \\6 m , x + fo\\d\\ 



(101) 



where /3% is defined in d22l . and /3 2 and fa are defined in ( 1471 ). 
The boundedness of 0(f) follows from Theorem |2"T1 □ 

Appendix G 
Proof of Theorem[9] 

Proof: The dynamics of the CRM and the ORM are given 
in (fJJ and ( TTOb respectively and leed to the following 



x m (t) - x° m (t) = A m (x m (t) - x° m {t)) - Le. 



(102) 



Given that the reference model will have the same initial 
condition regardless of being closed or open, we then have 
that 

\\x m {t) - x° m {t)\\ <mj exp(--(t-T))ie(r)dT (103) 
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where the matrix exponential bound came from ( f89b . Using 
the Cauchy-Schwartz inequality we have the following bound 



frn 

\\x m (t)-x° m (t)\\<^=\\e(t)\\ 



Appendix H 
Proof of TheoremQT] 

Taking the time derivative of u in (0 

u(t) = - b T Pe(t)x T (t)jI nxn x(t) 

+ 6 T (A m x(t) + b (e T x{t) + r(t))) + r(t) 



(104) 



(105) 



Substitution of the upper bound on P from H5i . using the 
definition of ag from (133b and the bounds on the reference 
trajectory from Assumption [3] results in the following bound 

|4(*)l<^l|6||||e(*)||||*(*)|| a (106) 
+ fi'max (ae\\x(t)\\ + r ) + n. 

A. Proof of Theorem\ll\ t E Ti 
The following Lemma is useful: 

Lemma 28. [Finite time stability] If r satisfies Assumption^ 
then 



\\x{€)\\ < || e (0)||expM) 



ro 



{exp(agt) - 1) , t > 
(107) 



where ag is defined in ( 133b . 

Proof: Suppose z(t) e I is defined as the solution to 

i(t) = a e z(t) + \\b\\r . (108) 
It can be shown that if z(0) = ||x(0)||, then 

\\x(t)\\ < z{t) Vt>0 (109) 

□ 



using 1291 Theorem 8.14]. 

Using the fact that .t(0) = e(0) which follows from 
Assumption HI Lemma [28] and the definitions of ag and T\ 
we obtain that 

Bup||x(t)|| <G xA (110) 

i£Ti 



where G x ,\ is defined in 

Beginning with (1251) . taking the square root of the expression 
and noting that y/a + C2 < ^Jc{ + ^Jc2 for all ci , C2 > 0, we 
obtain 



IK2 ^ 



\\e(t)\\ < ^e^(-M) || e (o)|| + y — „„ 



where t\ is defined in 



This verifies that 



sup||e(t)|| < G e .i 

teTi 



(111) 



(112) 



where G eA is defined in (J36j . Using ( fT06] i, (fTTOt . and (fTT2l . 
Theorem [TT] for t G Ti is proved. 



B. Proof of Theorem\TJ\ t £ T 2 
From (11 lit it is easy to see that, 

sup ||e(t)|| < G e , 2 

t>Nn 



(113) 



where G e ^ is defined in 

From (O and the bound on exp(A m t) in d89l . we have that 

\\x m (t)\\ < m |* C xp (-±(t r)) (Z||e(r)|| + ||6||||r||)dr 

(114) 

Using the integral transform of LTI systems, the bound for 
exp(^ m ) from ([89]l, the bound for ||e(t)|| from ( II 1 lb . (11141 ) 
takes the form 

||a^(t)|| <mi||e(0)|| (cxp(-^t) - cxp (-£*)) 
2Zm l2(s + £)~ 



■Mm 2(8 + 1 2 

— W— — ^(l-exp^-t 



2||6||m / / i 

+ ^^-ro(l-exp(^-if 



(115) 



2//, 



where mi - 

Given that x = e + x m , using dl 121 ) and dl 151 ) one can 
conclude that 



sup ||x(t)|| < G^.2 
t>Nn 



(116) 



where is defined in d36t . Using ( 1106b , (11131 ), and d 1 1 6b . 
Theorem [TT] for t € T2 is proved. 



C. Proof of Theorem [77] t G T3 

G e 3 follows from Corollary [2] G x 3 follows from ( 1115b . 
where it is noted that f > A^T2, and the fact that ||a;|| < 



Appendix I 
Proof of Theorem[T31 

The bound given for x(t) and e(t) over the time period 
[0, Nt*] in (1107b holds regardless of the choice of I. Thus the 
bound in ( l35l l holds for £ = 0, and therefore the bound in (l42l 
for t g [0, Nt^} is the same as that in (|37| i where p has been 
replaced by po. The Gronwall-Bellman analysis used to obtain 
the bound for e(t) would follow with a similar bound to that in 
(11 1 lb where the exponent would now have the time constant t' 2 
with e(t) exponentially decaying to \/l/Po^max. For t > Nt 2 , 
||e(t)|| would have decayed past 4 time constants. Therefore, 
the coefficient G e .2 would apply for the ORM case when 
t > Nt 2 and I = 0. The bound for x(t) would not contain 
the parameter I. Therefore, d42l for t > Nt 2 is identical in 
structure to d37b for t > Nti with £ = and p being replaced 
with po- The asymptotic properties of the adaptive system hold 
regardless of the choice of £ and therefore Corollary [2] holds 
when I = as well and thus the bounds in d42l for t > T{ 
hold as well. 
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Appendix J 
Proof of Theorem[T51 



Proof: Taking the time derivative of V in d53) results in 

(117) 



V < ~\\e m \\ 2 - \\eA\ 2 - 2^e 



7 



Substitution of V in d53l results in the bound in (154) . Using 
the bound in Lemma E]-(ii) we have that 



p T P p > 



"e m \\ 2 and ej Re, > 



2(s + £) 



2(a + £) 



M 5 



then we can conclude that lim t _ i . 00 ||e m (f)|| 2 < /340, 2 ax and 
limt^oo || ei(t) || 2 < /Js^max where p4 and /3s are defined in 
(|55) . The boundedness of 0(f) and 0(f) follows from Theorem 
l24l The asymptotic limit to zero comes from the application 
of Barbalat Lemma. □ 



Appendix K 
Proof of Theorem 

The bounds in (156) and (T57T > follow from the application of 
Gronwall-Bellman to the result in ( 154) with the lower bound 
for min Ai (P) in ( [ToT l and the change of parameters from 
being used. 

Beginning with 



||e m (*)||£ 2 < / -V(e(f), 0(f)) <V(e(0), 0(0)) 
Jo 

777 ^ 1 

< -l|e m (0)|| 2 + ^||e, : (0)|| 2 



■a + 2t 



2{a + 1) ' 



(118) 



-ll^(o)|| 2 , 

7 



using the definitions of p from d23) . the fact that ^-p^ < 
the bound in ( f58l > holds. This same approach can be used to 
obtain the bound in 



Appendix L 
Proof of TheoremPT"91 

Proof: Taking the time derivative of V in d69) results in 

V<-(1- A(£)) (||e m || 2 + ||e || 2 ) - 2^e 2 . (119) 

7 

where A(l) is defined in (1681 . Substitution of V in (|69) results 
in 

U<-a 5 ^ + a 6 (120) 

where a$ and «6 are defined in (T7TT >. Using the bound in 
Lemma |5]-(ii) we have that 



p T Pp > 



"e m f and eJPe > - 1 - ||e || : 



2(s + 1) 



2{s + £) 



then we can conclude that lim t _ ! . 00 ||e m (f)|| 2 < /?6^max an( l 
liixii^oo ||e (i)|| 2 < PeOi^ where /3 6 is defined in (72). The 
boundedness of 0(f) and 0(f) follows from Theorem l24l □ 



Appendix M 
Proof of Theorem 1201 

Proof: Taking the time derivative of V in (169) results in 

U<-(l-AW)(||e m || 2 + ||e || 2 )-2^e 2 

7 . (121) 

+ 2||P||||n(f)||||e m (f)||+2||P||||n(f)||||e (f)|| 



completing the square in ||e m ||||n|| and ||e ||||n 

II 2 i ||„ 1 1 2 



V < 



(1-A(£)) 





2 v 


(1 


- m) | 






(1 


- AW) | 
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(i-AM) 

4 



z— e e 

7 

|P||||n(f)| 



" (1-A(I)) 

ni P|| 2 Hf)|| 2 . 



P||||n(f)|| 



(122) 



Neglecting the negative terms in lines 2 and 3 from the 
equation above and substitution of the norm for P we have 
that 



v<- {1 A(£)) (IMI 2 + IM 2 ) 



2 
10 



2^ 2 



\P\\ 2 \\n(t)\\ 



(123) 



a-A(i)r 

which in terms of U is identical to 

• (l_AWHo + 2£) (l-A(l))(a + 2l) 22 
2^ ^ max 



7m' 
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(1- A(£)) 2 V°' + 2 ^ 



U < — a 7 V + a s 



(124) 
(125) 



where a 7 and as are defined in (176) . Using 
the bound in Lemma |5]-(ii) we can conclude 
that limt-voo||e m (t)|| 2 < /3 6 2 ax + l3 7 \\n(t)\\ 2 and 
limt^lMf)!! 2 < /3 6 2 ax + P 7 \\n{t)\\ 2 where p 7 is defined 
in (77) . The boundedness of 0(f) and 0(f) follows from 
Theorem l24l □ 



